There is a technique for matching an image (projected image) obtained by projecting an image model with an image acquired by an imaging element and for displaying the projected image and the acquired image in a region for displaying the acquired image. The display of the matching projected and acquired images is executed by adjusting the position, size, orientation, and the like of the projected image in the display region based on the position, size, orientation, and the like of an image of a reference object included in the acquired image.
As the reference object used to adjust the display of the projected image, an identifier that enables relative positions and relative orientations (relative angles) of the reference object and imaging element to be measured based on the image of the reference object within the acquired image is used. For example, an identifier that is referred to as an augmented reality (AR) marker is used as the reference object. The AR marker is, for example, a printed object on which a design (pattern) from which the relative position and relative angle with respect to the imaging element are measured based on an image obtained by imaging the AR marker itself is printed, or the like. In addition, there is an AR marker from which information may be acquired by analyzing a pattern printed on the AR marker based on an image of the AR marker within an acquired image.
During the display of the projected image, when at least either the relative positions or relative angles of the reference object and imaging element change due to the update of the acquired image, the updated acquired image and the projected image may not match each other. Thus, the projected image is readjusted based on the relative positions and relative angles of the reference object and imaging element. In the readjustment of the projected image, if the image of the reference object is not included in the updated acquired image, the position and size of the displayed projected image to be subjected to the matching with the updated acquired image are unclear and thus control is executed so as to terminate the display of the projected image. As described above, the display of a projected image that may not match the acquired image may be terminated by the control executed to display the projected image based on determination of whether or not the image of the reference object is included in the acquired image.
Regarding redrawing of a projected image, there is a technique for redrawing a projected image only when a value of an orientation sensor installed in a terminal device is equal to or larger than a predetermined value in a state in which the projected image is displayed in a display region. These techniques are disclosed in Japanese Laid-open Patent Publication No. 2010-238096, Japanese Laid-open Patent Publication No. 2000-156849, and Japanese Laid-open Patent Publication No. 2006-18818.